Low power fingerprint capture system, apparatus, and method

ABSTRACT

The present invention provides a large format fingerprint capture apparatus, system and method that is low power, compact, and lightweight and has a platen area greater than 3.0 square inches. The present system is typically powered, controlled, and exchanges data over a single data/control/power connection to a host PC, e.g., a desk top computer, PDA, or laptop computer although the system can also be used in a wireless fashion with a power subsystem so no physical connections are required.

CROSS REFERENCE TO RELATED APPLICATION

This patent application is a continuation of U.S. patent applicationSer. No. 12/782,620, filed May 18, 2010, which is a continuation of U.S.patent application Ser. No. 11/030,327, filed Jan. 7, 2005, which claimspriority to U.S. Provisional Patent Application Ser. No. 60/534,394,filed on Jan. 7, 2004. This application is related to U.S. patentapplication Ser. No. 12/194,359, filed Aug. 19, 2008. Theabove-identified applications are all hereby incorporated herein byreference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a system, apparatus, and method for thecollection of friction ridge signatures from a subject. Moreparticularly, the present invention relates to a low power consumption,small size, and low weight friction ridge capturing device and methodfor the collection of friction ridge signatures from a subject. Mostparticularly, the present invention relates to a low power consumption,compact, and portable digital friction ridge capturing apparatus, systemand method for the collection of friction ridge signatures with a platenarea of at least 3.0 square inches.

2. Description of the Related Art

Human beings have friction ridges on their hands and feet. Frictionridge impressions from a subject's fingers are commonly known asfingerprints. Animals also commonly have unique friction patterns ontheir footpads. In dogs and cats, for example, these patterns are calledpaw prints.

Digital scanning systems that capture friction ridge impressions,collectively termed herein as ‘fingerprints’, are old in the art. Manyof these systems were designed to capture a smaller area of one or twofingerprints while others were designed to capture a much larger area.Such existing systems commonly use optical imaging, capacitance,infrared radiation, ultrasound, or other means to capture fingerprints.

Many optical imaging systems are designed to capture one or twofingerprints at a time. These systems are termed ‘single print devices’herein. For example, the manufacturers listed in Table 1 provide opticaldevices that scan one or two fingerprints at a time.

TABLE 1 Name Web Address Cross Match Technologieshttp://www.crossmatch.com Exact ID http://www.exactid.com/ Identixhttp://www.identix.com/ Secugen http://www.secugen.com

Devices that capture a single fingerprint at a time are compact and drawminimal power during operation. One of the first issued patents thatdiscloses how such a single digit device works is U.S. Pat. No.3,200,701 to White, the entire contents of which is hereby incorporatedby reference as if fully set forth herein. White teaches a device thatuses a light source, a prismatic body, the principle of Total InternalReflection “TIR”, and a scanning apparatus to capture a fingerprintimage. A typical TIR scanning system that captures fingerprint imagescomprises a light source, a prism, a camera, and a host computer orother control device that is used to capture the image. It should beunderstood that the camera is any type of suitable sensor for thecapture of image data. This includes Charge Coupled Devices (CCD) andComplimentary Metal Oxide Semiconductor (CMOS) cameras as well as sensorchips included in these cameras and both linear and area scan versions.The host computer or other control device is referred to as a hostcomputer. There may be other components of a prior art system such as,e.g., polarizing filters, corrective optics, and holographic gratings.

Most commercially available large format optical systems today followthis single digit system configuration. That is, they use a lightsource, prismatic body, TIR, camera(s), and host computer to createfingerprint images. For prior art devices, “capable of capturing morethan two fingerprints simultaneously” means optical devices having asurface capture area exceeding 3.0 square inches. This type of system isreferred to as a large format fingerprint capture system. In addition,large format fingerprint capture systems include those that capture palmprints and writer's edge.

Large format fingerprint devices typically capture fingerprints frommultiple fingers simultaneously and therefore, the area upon which thesubjects place their fingers must be large enough to accommodate themaximum number of fingers to be captured simultaneously. Usually, thisnumber is four, but Cross Match Technologies provides a system that isable to capture the fingerprints in two groups of two fingerprintsapiece. In effect, this Cross Match Technologies system captures fourfingerprints simultaneously.

Livescan systems, one common form of large format fingerprint system,typically use a glass or plastic surface, termed a platen, upon whichthe subject's fingers are rolled or pressed. Images of the fingers'ridges are typically captured from underneath the platen by one ormultiple cameras and are then converted into digital files. Images ofrolled fingers are called rolled prints and images of pressed fingersare called slaps. Livescan devices are available from the sources listedin TABLE 2.

TABLE 2 Name Web Address Cross Match Technologieshttp://www.crossmatch.com Heimann Biometric Systemshttp://www.hbs-jena.com/ Identix http://www.identix.com/ Printrakhttp://www.printrakinternational.com/

A large body of patents exists for large format fingerprint scanningdevices. U.S. Pat. No. 3,200,701 to White, discussed above, disclosesone such system. Further, U.S. Pat. No. 4,933,976 to Fishbine et al.,the entire contents of which are hereby incorporated by reference as iffully set forth herein, teaches a configuration comprising aprismatic-based TIR device for fingerprint capture. Fishbine et al.disclose a method that combines successively captured images into oneimage array.

U.S. Pat. Nos. 5,548,394, 5,629,764, 5,650,842, 6,178,255, and 6,407,804all disclose variations of the TIR based prismatic platen device used tocapture fingerprint images.

In U.S. Pat. No. 5,548,394 to Giles et al., the entire contents of whichare hereby incorporated by reference as if fully set forth herein,teaches a TIR based system that uses a linear CCD camera (as opposed toan area based camera) and associated optics that is used to createrolled fingerprints. Prints are captured by changing the orientation ofa minor as a scan progresses.

U.S. Pat. No. 5,629,764 to Bahuguna et al., the entire contents of whichare hereby incorporated by reference as if fully set forth herein,teaches correcting the aspect ratio of images using holographicgratings. Light incident on the fingerprint surface is totallyinternally reflected at the surface of the prism but immediately afterbeing reflected back toward the prism the light enters a holographicgrating that changes the light direction. Light, directed now at thehypotenuse of the prism at an angle greater than the critical angle, istotal internally reflected toward the camera. In this patent, the aspectratio is corrected before the light leaves the prism.

U.S. Pat. No. 5,650,842 to Maase et al., the entire contents of whichare hereby incorporated by reference as if fully set forth herein,discloses a fingerprint capture system that optically corrects theaspect ratio of images after the image beam leaves the prism. Correctionis achieved by including optical elements in the image path after theimage exits the prism. The image may be generated using TIR or lightdispersion from the friction ridges. A reference light source is an edgelit light panel manufactured by Hewlett-Packard that is illuminated byred Light Emitting Diodes (LEDs).

U.S. Pat. No. 6,178,255 Scott et al., the entire contents of which arehereby incorporated by reference as if fully set forth herein, disclosesa method and apparatus in which a mechanism slides a prism over animaged area of a camera. By using a linear encoding mechanism, themethod and apparatus splices together complete fingerprint images fromthe smaller portions of the fingerprint that are captured when slidingthe prism over the camera.

U.S. Pat. No. 6,407,804 to Hillman et al., the entire contents of whichare hereby incorporated by reference as if fully set forth herein,discloses a fingerprint capture system with embedded reference targetsin the optical path. Using these reference targets, the system can becalibrated at anytime using these internal reference targets. Thispatent also discloses use of a planar light source constructed from atwo dimensional array of LEDs followed by a diffusor.

Several patented systems do not teach that TIR be used at thefingerprint imaging surface in order for a fingerprint to be captured.Such systems are disclosed in U.S. Pat. Nos. 5,621,516, 5,650,842 and6,061,463.

U.S. Pat. No. 5,621,516 to Shinzaki et al., the entire contents of whichare hereby incorporated by reference as if fully set forth herein,discloses a system that images randomly reflected light from thefriction ridges. By providing means of minimizing the light contentreflected by platen areas not in contact with the friction edges, thecontrast of the resulting images are improved. U.S. Pat. No. 5,650,842to Maase et al., discussed above, provides another mechanism for captureof friction ridges via light dispersion.

U.S. Pat. No. 6,061,463 to Metz et al., the entire contents of which arehereby incorporated by reference as if fully set forth herein, teachesusing a slanted-fringed light diffractive grating to redirect lightperpendicularly toward the platen surface. Light reflected from thesurface is then captured by the camera. Light incident upon the frictionridges is dispersed and therefore this less intense reflection is imagedas dark. One disclosed embodiment uses volume holograms to redirectlight. This patent contrasts with U.S. Pat. No. 5,629,764 to Bahuguna etal. in that this patent redirects the light before it reaches the platensurface and therefore image aspect correction is never required.

Based on the foregoing discussion, most, if not all, existing commerciallarge format fingerprint devices use the principle of TIR. In themajority of these devices, the object plane to be imaged (thefingerprint) and the image plane of the camera are not parallel,centered, and perpendicular to a common axis. To correct for opticalperspective distortions introduced by the relative positions of theobject and image planes optics within the device must correct thepositions of the object and/or image planes before the camera capturesan image or the system must employ an algorithmic solution forcorrecting the perspective distortion introduced into the image. In thefirst case, if additional optical components are added, the size andweight of the device increase. For example, see U.S. Pat. Nos. 5,650,842and 6,407,804. In the later cases, to avoid an unfocused image the depthof field must be deep enough for the entire object area to be in focus.Since the image is not optically corrected for perspective distortion,the depth of field requirement is driven by the three dimensionalgeometry of the platen, the optics used between the platen surface andthe camera, and the geometric relationship between the camera and theplaten surface. Typically, lenses that allow larger depths of field havefocal lengths such as 30 mm or greater with corresponding f-stops oftengreater than 4.0. Long focal length lenses often result in a distancefrom the object plane to the image plane that is too large to put theentire device into a physically compact solution. In addition, highf-stop lenses restrict the amount of light entering a camera andtherefore more light, power, and/or exposure are needed for thesesystems, thus implying a larger power consumption.

Often aberrations in the periphery of the image such as barreldistortion and pincushion distortion distort the fingerprint to thepoint where the images are not truly usable. Barrel distortion occurswhen the distance of the pixels along the image edges are farther awayfrom the center of the image than the corresponding actual distance inthe target. Pincushion distortion occurs when the pixels along theimages edges are closer to the center of the image that thecorresponding actual distance in the target. Barrel and pincushiondistortions are introduced via lenses used in an optical system and bothdistortion types may be present at the same time.

The majority of existing optical fingerprint systems rely on LEDs forlight sources. Light illuminating the platen may be either diffuse orcollimated.

Electricity consumers in a fingerprint device include the lightsource(s), the camera(s), frame grabber electronics, magnetic stripereaders, barcode readers, radio frequency identification modules,proximity card readers, smartcard readers, displays, platen heaters,platen blowers, and servomotors, if present. In total, the power used bysuch systems is above 10 watts for all existing large format fingerprintsystems. Therefore, prior art large format fingerprint devices arepowered by external power sources connected via separate cabling sincepower provided over a single data/control/power cable will either beinsufficient or the battery on the attached computer will be drained tooquickly. In other words, prior art systems cannot be powered only fromthe computer to which the device is attached.

Most of the patented systems described above are either not compact ornot lightweight and therefore they cannot be considered as portable. Tobe moved, these devices often require a protective case. In existinginstances, the device and case weigh over 30 pounds. In addition, manydevices must be re-calibrated once the device has been moved andreinstalled. Such re-calibration is required due to the presence ofmoving parts or the possibility that parts have moved relative to oneanother.

Such systems also commonly address the issue of condensation on theplaten. Such condensation occurs when the dew point around theplaten/finger is too high relative to the ambient temperature of theprism and therefore moisture from the finger condenses on the prism. Insuch cases, platen heaters and platen blowers have been used to minimizethe condensation effects to the image.

No prior art large format fingerprint scanning device has the ability topass all data, power, and control logic over a single physicalconnection to the device. In addition, image-processing means toidentify the start and stop points of a fingerprint image capturesession also do not currently exist. Rather, external controls such asfoot pedals, touch screens, keyboard keys, and buttons located on thedevice are commonplace as means to identify start and stop points forcapturing fingerprint images.

SUMMARY OF THE INVENTION

Thus, there is a need for a fingerprint capture device that combines thefeatures of small size, low weight, low power consumption, andincorporates non-external fingerprint start-stop scanning control. Thesystem, apparatus, and method of the present invention preferablyprovides these features by:

1. balancing depth of field and lens focal length requirements;

2. algorithmically correcting image aberrations on the lens peripheryvia hardware, firmware or software;

3. algorithmically correcting perspective image distortions viahardware, firmware or software;

4. generating fingerprint images using minimal power, at most about 3.0watts, preferably at most about 2.5 watts, while being able to meet theelectrical requirements of the electrical components (such as lightsource, camera, magnetic stripe reader, radio frequency identification(RFID) module, proximity card reader, smartcard reader, platen heater,platen blower, and barcode reader), e.g., the camera requires at mostabout 2 watts and preferably at most about 1.8 watts;

5. minimizing the number of internal components as well as their weightand size so that the device weighs at most about 10 lbs and has a volumeat most about 400 in³ and preferably weighs at most about 7 lbs, morepreferably less than 5 lbs and preferably has a volume less than about325 in.³, e.g., 9 in.×7 in.×5 in.=315 in.³ or at most about 4.7 in.×2in.×16 in.=240 in.³;

6. building a device interface utilizing at most a single cableconnection that combines data, control, and power (e.g., USB, FireWire,Ethernet); and

7. enabling capture of fingerprint images using image processing of theimages themselves as capture start and capture finish signals (autotriggers).

The present invention overcomes the deficiencies of prior art largeformat fingerprint devices by providing a fingerprint capture apparatus,system and method that is low power, compact, and lightweight and has aplaten area greater than about 3.0 square inches, e.g., from about 3 toabout 24 square inches. Further, the present invention is typicallypowered, controlled, and exchanges data over a single data/control/powerconnection to a host PC. In an alternative preferred embodiment thelarge format fingerprint device is directly connected to a completelydisconnected (not plugged in to a wall power outlet or other externalpower source) portable PC, such as a laptop having a battery powersource. In another preferred embodiment a wireless interface (e.g.infrared, 802.11b, Bluetooth, etc.) to the device exchanges data andaccepts control functions from an attached computer processor while thedevice runs on a completely self-contained power subsystem such as abattery. In such an embodiment the device may have no physicalconnection ports. If desired, the device exchanges data and acceptscontrol functions via internet and/or satellite connectivity to acomputer processor.

In some embodiments the device may have an internal computationalcapability so it can direct fingerprint capture and fingerprint matchingwithin the device using fingerprint templates that are delivered over awired or wireless network attached to the device. Typical applicationsof such devices would be access control and tracking applications. Thematching templates and algorithm could be updated over the electronicinterface.

The primary device components of the present invention combine tominimize required power, size and weight. The device of the presentinvention comprises a light source, a prism, a camera (including thelens), a housing, and a host computer. Optional elements compriseholographic elements such as gratings and holographic optical elements(HOEs), a battery subsystem, an image processing subsystem, opticalfilters, magnetic stripe reader, RFID module, proximity card reader,smartcard reader, barcode reader, a platen heater, a platen blower, andminors used to divert the image beam.

To achieve minimal size and weight the number of components is minimizedin the system, apparatus, and method of the present invention.Technology for aspect ratio changes, image aberration corrections, andperspective corrections may be used to minimize the depth of field, astaught by U.S. Pat. Nos. 5,629,764 and 6,061,463 incorporated herein byreference. Alternatively, aspect ratio, image aberration, andperspective corrections can be made algorithmically via hardware,firmware and software as provided for in the present invention. Ojanen,see Appendix A, teaches one way of removing image aberrations andperspective distortions via such algorithms. Also, technology may beused to maximize the amount of light generated per watt, as taught, byU.S. Pat. Nos. 5,359,691 to Tai et al., 5,390,276 to Tai et al., and5,854,872 to Tai, which are hereby incorporated by reference as if fullyset forth herein.

As disclosed in U.S. Pat. Nos. 5,629,764 and 6,061,463 incorporatedherein by reference, technologies to reduce the required depth of fieldto near zero exist. These technologies typically work by redirecting thelight in the device with holographic gratings or other suitablereplacements. Any optical technology that allows the object plane andimage plane to be properly aligned suffices but, the added advantage ofthe holographic elements as disclosed in U.S. Pat. Nos. 5,629,764 and6,061,463 is that they take a small amount of space. Since the objectplane and image plane are properly aligned, there is no need for heavierand more space consuming optics to perform optical image correction.

An alternative way to improve the depth of field without redirectinglight or adding optical components is provided by this invention. Sincethe prisms used in these devices are formed from planar surfaces, a realobject placed onto the planar platen surface can be described as aplanar virtual object that appears closer to the camera than the realobject. The plane orientation of the virtual object can be used inconjunction with the lateral magnification to generate an angle at whichto orient the sensor chip in the camera so as to minimize the requireddepth of field.

More specifically, in referring to FIG. 10A, a simple optical system hasan object 1005, a lens 1001, a focus 1002, and an image 1006. Thedistance along the optical axis from the principal point to the objectis the object distance 1004, o, and the distance along the optical axisfrom the principal point to the image is the image distance 1003, i. Aoptical system's lateral magnification is defined as m=i/o. Referringagain to FIG. 10A, if a planar object is imaged and that planar objectis rotated by angle α_(o) 1007 from the normal to the optical axis, thisplanar object appears as a real inverted image oriented with angle α_(i)1008 where α_(i)=tan⁻¹(m*tan(α_(o))) with respect to the optical axisnormal. The angle α_(i) is defined by α_(o) and m.

Now referring to FIG. 10B, the angle θ 1009 and the index of refractionn are properties of the prism 1010. In an imaging system such as FIG. 2and FIG. 10B, a real object 1005 at the prism surface is imaged as avirtual object 1012. In FIG. 10B, the virtual object appears at an angleα_(v) 1011 where α_(v)=tan⁻¹(tan(θ)/n). Consequently, given an idealoptical device setup, α_(v) is known and therefore

α_(i)=tan⁻¹(m*tan(θ)/n).

If the image sensor plane is placed at angle α_(i) in the ideal systemsetup, the depth of field will be zero. For example, if m=1/13, θ=45degrees, and n=1.52, then α_(i)=2.89 degrees.

Collimation or semi-collimation of the light source increases theefficiency at which the device uses light. Usage of more collimatedlight means that less overall light is needed for the camera to generateimages. The higher the overall light efficiency, the lower therequirements for power and light size. U.S. Pat. Nos. 5,359,691,5,390,276 and 5,854,872, all disclose compact light pipes that utilizemicroprismatic structures to guide light in a semi-collimated manner.These light pipes efficiently convert diffuse light from a Cold CathodeFluorescent Light (CCFL) into semi-collimated light.

After an image has been captured, it may need to have perspective,barrel and pincushion distortions removed. Typically, barrel andpincushion distortions are removed first, then perspective distortion isremoved. Optics have traditionally served this purpose. But, in thisinvention, barrel and pincushion aberrations can be removed with analgorithm developed by Ojanen. A pre-print describing this algorithm andsoftware that implements the algorithm may be found in Appendix A and athttp://www.math.rutgers.edu/˜ojanen/, the entire contents of which arehereby incorporated by reference as if fully set forth herein. TheOjanen algorithm relies on the capture of a high accuracy image of afixed array of dots of a particular size. By knowing the size andlocations of the dots within a grid, the systematic discovery of barreland pincushion distortions can be characterized by running a leastsquares algorithm on the error between the observed dot locations andsizes and the known dot locations and sizes. Correction of thedistortions is then completed using the returned least squaresparameters. Typically, the present invention employs a perspectivecorrection algorithm in addition to the Ojanen algorithm.

In the present invention, depth of field issues in some embodiments donot drive the lens requirement so a shorter focal distance lens can beused at the expense of potentially increasing aberrations. But, sincesuch aberrations are predictable, they can be reasonably corrected usingsoftware that implements the Ojanen algorithm. Another important sideeffect of a shorter focal length lens is the f-stop setting. Since a lowdepth of field is required, a lower f-stop lens is used so that moreefficient use is made of the light and thus size and power requirementsare further reduced.

The system power requirements of the present invention are reduced tothe point that the device can be run within the limits of the powersupplied via common computer interfaces such as FireWire and USB, e.g.,USB-2, (FireWire can provide up to 50 watts of power and USB up to 2.5watts). The unanticipated and non-obvious innovation of the presentinvention is enabling the fingerprint device to be powered by acompletely disconnected laptop computer while maximizing the batterylife of the laptop and thereby extending the useful amount of time thesystem can be used. The invention provides for a device that consumes atmost about 3.0 watts of power, preferably at most about 2.5 watts ofpower. Typically, the light uses at most about 1 watt, preferably atmost about 0.7 watts and the camera uses at most about 2 watts,preferably at most about 1.8 watts.

In the device, apparatus, and method of the present invention, as onealternative to LED light sources, a CCFL is used. Other light sourcesinclude electroluminescent sources and lasers. The power source for thislight is an electrical inverter that taps power off from the poweroriginating from the host computer interface. A single CCFL is used toilluminate a light pipe that partially collimates the light beforesending the light into a prism. This CCFL and light pipe construction,as described in U.S. Pat. Nos. 5,359,691, 5,390,276, and 5,854,872generates enough light at a low power to serve as the system lightsource for the apparatus, system and method of the present invention.The rated lifetime of such CCFL's is about 10,000 hours of operation. Aswith LEDs, the rated lifetime is the amount of time the light is on atthe rated power until the light output is one half of the originaloutput. Not only does the light source emit enough light for theapparatus, system and method of the present invention, it also isdelivered in a very compact size and thus contributes to the size of theapparatus, system and method of the present invention.

In the apparatus, system and method of the present invention, twoelectrical loads exist in every embodiment: the light source load andthe camera load. In a preferred embodiment, the light source loadcomprises the electrical inverter and the CCFL. In alternativeembodiments, there are at least several other loads: an optionalelectrical storage device, a magnetic stripe reader, RFID module,proximity card reader, smartcard reader, and a barcode reader. Inalternative embodiments, a battery, solar cell or capacitor subsystem isused to supply electrical energy to system components. In particular,such a subsystem is needed in the case where more power is needed thanthe computer interface can provide. This can be the case, for instance,in an embodiment comprising at least one of a magnetic stripe reader, anRFID module, a proximity card reader and a smartcard reader to readdemographic data from the back of a driver's license and an embodimentcomprising a one dimensional or two dimensional barcode reader to readthe demographic data from the bar code on the back of a driver'slicense.

In alternative embodiments, images are captured with either aone-dimensional (line scan) or two-dimensional (area scan) camera.Preferred embodiments comprise area scan cameras to increase systemrobustness by avoiding the use of moving parts. In these embodiments,the image generated must be large enough to capture the entire objectarea at a prescribed resolution. Two such cameras with USB 2.0interfaces are the Silicon Imaging SI-3170-U and the Silicon ImagingSI-6600-U cameras.

In a preferred embodiment, the camera uses the light provided by thelight pipe to capture images of rolled and slapped impressions offingerprints. The electrical interface to the camera also comprises thecontrol signals that operate the camera, operate the magnetic stripereader, operate the barcode reader, operated the RFID module, operatethe proximity card reader, operate the smart card reader, and turn thelight source on and off. That is, in a preferred embodiment, all power,control, and data to be exchanged between the host computer and thefingerprint device are exchanged via the single connection between thecomputer and the device.

An important aspect of the control logic to capture fingerprints is howto determine when to start capturing a print and when to stop capturinga print. In preferred embodiments of the apparatus, system and method ofthe present invention, this control logic is implemented in softwaresince the frame rate of the images delivered by the camera is highenough to allow processing on the host computer that algorithmicallyidentifies a starting frame and an ending frame for each fingerprintcapture. Typically, for a single finger roll the real frame rate is 20frames or more per second, which results in a 12-13 or more processedframes per second rate. Typically, for a four finger slap image the realframe rate is at least 6 frames per second which results in an at least4 processed frames per second rate. As a rolled fingerprint is beingcaptured frame-by-frame, individual frames are analyzed and combinedinto a rolled print so that when the ending frame is identified, thecombined rolled image is complete. Preferably, the frame rate for arolled image is sufficient to obtain a real-time image.

In summary, in a preferred embodiment, the apparatus, system and methodof the present invention provide a light source and camera combinationthat has a power and light efficiency that allows a large formatfingerprint device to be powered, controlled, and to exchange datadigital image frames over a single connection, such as a USB 2.0 cableconnection. Alternative embodiments include FireWire 1.0, FireWire 2.0and next generation peripheral interfaces. Alternative embodiments alsoinclude a power subsystem wherein a battery or capacitor is chargedduring periods of low power consumption and when more power is requiredmore power is drawn from the power subsystem.

In order to ensure continued operation of the apparatus, system andmethod of the present invention, non-volatile memory is included insidethe device so that statistical and diagnostic data can be collected andmonitored. In a preferred embodiment, the number of times the lightsource switches on and off is maintained in non-volatile memory so thata predetermined maintenance schedule can be followed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a light source that can run on low power efficiently;

FIG. 2 illustrates a large format fingerprint device according to thepresent invention;

FIGS. 3A-B illustrate a large format fingerprint device in which aholographic grating has been incorporated according to an embodiment ofthe present invention;

FIG. 4 illustrates a large format fingerprint device with either abattery subsystem or a capacitor that powers the electrical consumers ofthe device, according to an embodiment of the present invention;

FIGS. 5A and 5B illustrate a large format fingerprint deviceincorporating a two-dimensional barcode reader with imaging capability,according to alternative embodiments of the present invention;

FIG. 6 illustrates a large format fingerprint device incorporating amagnetic stripe reader, according to an embodiment of the presentinvention; and

FIG. 7 illustrates an embodiment of the present invention comprising adevice-resident computer processor and a device-resident non-volatilememory.

FIGS. 8A-D are a flow diagram of the method of the present invention.

FIG. 9 illustrates a slip case for covering the device.

FIGS. 10A-B illustrate how changing the angle of the image sensorminimizes depth of field.

FIG. 11A-E illustrates a preferred compact embodiment of the device.

FIG. 12 illustrates an alternative preferred compact embodiment of thedevice based upon the device in FIG. 11.

FIG. 13 illustrates a possible case for the device which can begenerated using an extrusion.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present invention focuses on the use of optical imaging to providean optical fingerprint scanning apparatus, system and method forcapturing more than two fingerprints simultaneously.

A camera as used herein is minimally the combination of a lens and animage capture capability. For digital cameras, such image capturecapability includes an image sensor and a frame grabber that convertsthe sensed image data to a digital format, e.g., an image frame. Forfilm cameras, the image capture capability is provided by film. Digitalcameras optionally have interfaces to transfer digitized images to asystem for processing.

An image frame as used herein is data output by the camera thatrepresents at least a portion of the scene, which the camera iscapturing

A capture sequence as used herein is a series of at least one imageframe provided by the camera from which at least one image frame isselected for generating an output composite image.

A physical connection port as used herein is a connector whereby a cableor other physical electronic communication mechanism is operativelyattached to the connector.

A roll capture sequence as used herein is a capture sequence used togenerate a composite rolled fingerprint image.

A slap capture sequence as used herein is a capture sequence used togenerate a slap fingerprint image.

Each of the preferred embodiments and alternatives thereof comprises alight source consisting of a light pipe providing semi-collimated lightor a collimated light source. FIG. 1, illustrating an embodiment of sucha light pipe, shows a backlighting assembly system using a linear lightsource such as a cold cathode fluorescent lamp (CCFL) 92. In thissituation, a beam expander 6 has a width approximately equal to thewidth of the platen of the fingerprint capture device. The beam expander6 expands the linear light source into a plane light source. As shown inFIG. 1, a minor reflector 100 is wrapped around the lamp 92 to collimatelight in one dimension. Divergent angle rotating elongated microprismstructure 16 is created on the top surface to rotate the light beams sothat output light is collimated in both dimensions. Microprisms 94located on the bottom surface are used to reflect light out. A side ofthe light pipe opposing the lamp is coated with a reflecting film 102 toreflect light back towards the microprism side and reflecting film 102may be made to tilt towards the bottom surface so that essentially allof the light will be reflected out by the microprisms 94.

First Embodiment

Referring now to FIG. 2, a first preferred embodiment of the presentinvention serves as a basic embodiment on which all other embodimentsare founded. The first preferred embodiment has especially low powerconsumption, consuming at most between 3.0 and 10.0 watts, preferably atmost about 2.5 watts, to lower battery drain on a host computer notconnected to an electrical outlet.

As illustrated in FIG. 2, the first preferred embodiment includes acamera 204 having a lens and optionally a filter 251, an efficient lightsource 201 that consumes at most about 1 watt, preferably at most about0.7 watt, and emits sufficient light for the camera 204 to obtain anacceptable image, a prism 202, an optional light control film 250interposed between said light source 201 and said prism 202, and aninterface 205 to a host computer 206. In this first preferredembodiment, light emitted by the efficient light source 201 enters theprism 202, is controlled by the light control film 250 and intersectsthe prism surface (platen) 207 at an angle greater than the criticalangle. Light intersecting the fingerprint ridges 220 is scattered orabsorbed while light that hits surface 207 of the prism with no ridgespresent is reflected. The reflected light then exits the prism 202 in adirection 208 towards an optional minor 213 that reflects the lightalong optical axis 214 in a direction towards the camera 204 andoptionally the filter 251. The optional mirror 213 is typically aprecision dichroic mirror so that the mirror can additionally helpremove ambient light that enters the system. The filer 251 substantiallyblocks ambient light from entering the camera. For example, if lightsource 201 emits a green light then minor 213 and filter 251 pass onlygreen light Likewise, if light source 201 emits a red light then minor213 and filter 251 pass only red light. Alternatively, filter 251 canalso filter infrared light. Some of the scattered light may also exitthe prism in a direction of the mirror 213. The camera 204 captures aframe and transmits the frame to the host computer 206. Typical lensesfor the camera have an f-stop of 3.0 to 8.5.

In each of the embodiments of the present invention, typically, anexternal stimulus causes a device according to the present invention toturn on the light source 201 before beginning fingerprint capture andturn off the light source 201 after the host computer is doneinteracting with the device. The preferred manner to control the lightis via a software control from the host computer 206. This host computer206 directs the device to change a switch that allows or disallowselectricity to flow to the light thereby turning the light on and off.

In each of the embodiments of the present invention, a record of countsand other data is typically maintained in a non-volatile memory 203 thatis located in the camera 204 electronics or elsewhere on the device.These counts are used to track the need for system maintenance. Suchcounts include, but are not limited to, amount of time the light sourceis on, the number of each type of fingerprint captured or rescanned, thenumber of times the light source was switched on, the number of timesthe light source was switched off, and the number of times the devicedetected that the light was off when it should have been on. Other datastored typically includes the device serial number, manufactured date,manufactured location, date and time latest used, and driver softwareversion number. A diagnostic software or firmware component typicallyinterrogates these counts in the non-volatile memory 203 to determine ifthe counts indicate device maintenance is needed. This diagnosticcomponent, in an alternative embodiment, is also configured to performtests to identify possible system errors. This diagnostic componentoutputs a diagnostic report based on the test results and the values ofthe counts. The diagnostic report can be viewed and browsed on a screen210 of an attached host computer 206 or can be printed and, in anyevent, can be stored in a persistent storage (not shown) by the hostcomputer 206.

The efficient light source 201 is preferably a CCFL using a light pipe(dimensionally flat, high and uniform output) or alternatively and LEDor other source providing a semi-collimated light source as adapted fromthe teachings of U.S. Pat. Nos. 5,359,691, 5,390,276, and 5,854,872 andother collimated light sources. The patents teach light sources thatinject light into the side of a light pipe. Microstructures within thelight pipe and the light guide connecting the light source to the lightpipe redirect the incident light into predefined directions. Thearrangement and geometry of the microstructures enable the light pipe tooutput light from the light pipe surface in a substantially collimatedfashion. The conical angles at which light leaves the surface of thelight pipe are predetermined by the arrangement and geometry of themicrostructures. The microstructures are typically microprisms. Thelight pipe configuration used for the present invention optionally alsoincludes a filter to restrict light emanating from the filter surface toprimarily semi-collimated light in a cone that diverges from the normalto the surface of the light pipe by approximately 30 degrees in eachdirection.

In an alternative embodiment an automatic feedback loop is used tocontrol light source intensity 201 since light output varies over time.This feedback loop is implemented by at least one of an optoelectronicfeedback loop for controlling input power to the light source 201 and adevice-resident or host computer 206 resident software for adjusting theexposure time of the camera 204. For instance, Microsemi, 2381 MorseAvenue, Irvine, Calif. 92614, sells silicon chips that can easily beincorporated into an optoelectronic light feedback loop.

The apparatus, system and method of the present invention preferablycaptures fingerprints using image processing operations as triggerconditions. For example, placing fingers of the subject on the platencould start a capture sequence, and the subject removing contact withthe platen could end the capture sequence or when a substantiallysimilar image occurs more than a pre-determined number of times couldend the capture sequence.

Alternative embodiments use foot pedals 212, buttons 211, keyboard keys(not shown), or touch screens 210. Image detection and captureprocessing operations are implemented in at least one of software,firmware, a dedicated circuit or a functionally dedicated chip. Theseoperations are implemented on at least one of the host computer 206, anetwork computing resource 219, a frame grabber (not shown) thatcaptures the frames, or within the device itself 200.

In preferred embodiments of the present invention, the interface 205 tothe host computer 206 is a single tether 205 that handles data, control,and power for the light source 201, camera 204, platen heater, platenblower, and optional devices. Optional devices include a barcode reader501 and a magnetic stripe reader 601. In preferred embodiments, theinterface comprises at least one of USB (USB-2) connection or FireWireand their variants or other interface for exchanging data and conveyingpower to operate. If the device of the present invention is plugged intoan external power source (not shown) such as a wall outlet or aninternal or external battery power source (not shown), the interface mayinclude Ethernet and its variants as well as optical fiber, suitable toenable high resolution images to be captured and transmitted forprocessing over the tether 205. Essentially, the interface can be anythat provides the interconnectivity between the capture device of thepresent invention and a target system that performs at least one ofreceipt of captured images, and processing of received images.

A protective covering (not shown) is provided which comprises one of acoating placed directly onto the device, a coating placed directly ontothe device combined with a removable cover, and a lightweight snap-oncarrying case that the device easily slips into and out of.

For a higher resolution camera such that scanning can be performed atleast at 500 dpi and 1000 dpi, the images captured can include at leastone of a single digit, up to 8 digits simultaneously, a palm print, awriter's edge, and all the slaps and rolls and footprints and noseprints required of an apparatus, system and method according to thepresent invention.

In each of the embodiments, camera lenses 204 may introduce imagingdefects such as barrel and pincushion distortion. The present inventionmay employ a suitable correction algorithm such as a correctionalgorithm substantially similar to that published by Ojanen, the entirecontents of which are hereby incorporated by reference, or othersuitable correction algorithm. A pre-print describing this Ojanenalgorithm and software that implements this algorithm may be found athttp://www.math.rutgers.edu/˜ojanen/ and is included in Appendix A. Thefirst step in applying the algorithm is to scan a reference target onthe device, which contains multiple geometric elements of known size andspacing with respect to one another. The size and spacing of thesegeometric elements is designed to capture barrel and pincushiondistortions. For example, a rectangular array of circles measuring 0.5mm in diameter and spaced 2.5 mm apart is used. Once the image of thetarget has been captured, the software attempts to automatically locatethe geometric elements in the image and attempts to form a rectangulararray of these elements. After a complete array of the geometricelements is identified, the software uses the known distance betweenelements, the size of the elements, and the center of the element arrayto measure differences (error terms) between the expected position ofthe elements and the actual position of the elements. These errors termsare used with a selected defect model to approximate the coefficients inthe defect model using a least squares type of algorithm. The outputcoefficients and the defect model are then used to correct a capturedimage using the Ojanen method. Correction in the image occurs after thefinal roll or slap has been captured.

Preferably, perspective distortion, if present, would be compensated foroptically, by firmware, by hardware or by software.

Second Embodiment

Referring now to FIGS. 3A-B, a second preferred embodiment of thepresent invention includes an efficient light source 201, a prism 202, aholographic grating 301 on an upper surface of the prism 202 and alight-transmitting substrate 302 on the holographic grating 301, acamera 204 having a lens, and an interface 205 to a host computer 206,and the host computer 206. The holographic grating 301, lighttransmitting substrate 302 and the upper surface 221 of the prism 202together form a platen. Optionally, the holographic grating 301 isattached by an adhesive to the upper surface 221 of the prism 202 and tothe lower surface of the light transmitting substrate 302 by an adhesive(not shown). Typically the holographic grating 301, light transmittingsubstrate 302 and the prism 202 are made of glass or acrylic polymer. Inthis second preferred embodiment, light emitted by the efficient lightsource 201 (light pipe) enters the prism 202 intersecting the prismsurface 221 at an angle greater than the critical angle. Then, the lightpasses through a holographic grating 301 and the light transmittingsubstrate 302 on the surface of the prism 202, hits the finger ridges220, and is scattered/absorbed or hits the surface of the substrate 302and is reflected 208. Reflected light 208 passes back through theholographic grating 301 and is corrected on its way back through theholographic grating 301. The scattered and reflected light then isfurther reflected by a surface of the prism 202 and exits the prism 202in the direction 208 of the camera 204. The camera 204 captures a frameand transmits the frame to the host computer 206.

In a preferred alternative of the second embodiment, the holographicgrating 301 is adapted from the teaching of U.S. Pat. No. 5,629,764. Theholographic grating 301 allows the size of the system to be reducedsince the depth of field requirement is now near zero. This translatesinto a lower f-stop and lower focal length lens, which in turntranslates into a shorter, required optical path length. In other words,the holographic grating 301 is significantly advantageous to thecompactness (and portability) of the present invention.

An additional alternative to the second preferred embodiment and theabove-mentioned alternative employs a holographic optical element (HOE)with a lensing function. Such an HOE serves a very similar function tothe holographic grating 301 but, in addition, it has a focusing abilitybuilt in so that part of the function of the lens can be off-loaded ontothe HOE. The net result in this additional alternative to both preferredsecond embodiments is that the optical path length can be made evenshorter than with the holographic grating 301 since the lens 204 can becloser to the prism 202.

Third Embodiment

The embodiment illustrated in FIG. 4 is substantially similar to that ofFIG. 3B but shows a power subsystem 401 based upon a Lithium ion battery402. In this subsystem, a Microsemi LX2201 chip, or similar chip, can beeffectively used to provide power to the electrical consumers in thesystem. If enough power enters the system through the tether 205, thispower is directed to the appropriate electronic components using aswitching implementation driven by software and firmware resident on thedevice 400. Extra power not used by the devices is used to charge thebattery 402 under the control of the software, firmware, or hardware.

In an alternative embodiment a capacitor 402 or a solar cell (not shown)replaces the Lithium ion battery 402. Such a construction can be usedwhen a power shortage is temporary and can be served by the capacity ofthe capacitor being used.

Fourth Embodiment

FIG. 5A illustrates a built-in barcode reader 501 that is a second imagecapturing device in the camera. The window 502 through which the barcodereader images documents (driver's licenses for instance) is placed sothat ambient light that may enter the system through the window does notinterfere with the fingerprint image being captured by the camera 204.This can be done several ways. In the preferred embodiment, the window502 is placed in a location behind a light blocker 503 that physicallyprohibits the light from disturbing the fingerprint image. In FIG. 5Asuch a location is behind the camera 204 that captures the fingerprintimage. In an alternative embodiment, since the barcode and fingerprintsare not typically captured simultaneously, a covering (not shown) isdesigned into the case so that the window 502 is covered when not inoperation. For example (not shown), the case may provide parallelgrooves flanking the window and a sliding cover is slidably located insaid grooves. The grooves are sufficiently long such that the cover mayslide from a first position which fully covers the window to a secondposition which fully exposes the window.

The power for the barcode reader 501 is tapped off of the powersubsystem 401. The 4100 unit from Hand Held Products, Inc. (HHP), 700Visions Drive, Skaneateles Falls, N.Y., 13153-0208, is an example of asuitable reader. Since the barcode reader 501 and the fingerprint imager204 are not operating at the same time, full power is only provided toone device at a time. The control logic that interfaces to the camera204 and the barcode reader 401 is written as firmware, in a preferredembodiment. This firmware communicates with the camera 204 and thebarcode reader 401 in their native formats. For instance, the HHPbarcode reader mentioned above communicates with a serial interface thatis implemented on a Silicon Imaging camera. The firmware within thecamera 204 manages this serial interface to control the barcode reader501. An external interface to the barcode reader 501 on the hostcomputer 206 simply talks to the firmware, which talks with the barcodereader 501.

FIG. 5B is an alternative embodiment of the barcode embodiment in whicha movable minor 504 is used to redirect the optical path of thefingerprint-capturing camera 204 through a secondary window 505. Whenthe mirror 504 is moved, manually or otherwise, into the appropriateposition the camera 304 can begin capturing images of scenes through thesecondary window 204. Captured images may be processed on the hostcomputer 206 with any number of barcode reading software packages. Afterthe barcode is read via this software, the minor 504 is placed back intoits original position and the operator can begin to capturefingerprints. In this embodiment, only a single camera 204 is being usedbut a second light source 506 must be provided. Such a light source caninclude LEDs or CCFL and the power to the second light source 506 can becontrolled as described above in the light pipe 201.

Fifth Embodiment

In a fifth preferred embodiment, illustrated in FIG. 6, a magneticstripe reader 601 is included in the device. Since these readers aretypically low power consumers and non-optical in nature, the magneticstripe reader 601 can be placed in any location that does not interferewith fingerprint capture. The magnetic stripe reader 601 draws powerfrom the power source 401 much as the barcode reader 501 and lightsource 201 do. The magnetic stripe reader 601 also has the option ofbeing turned on and off in software. Applications of the magnetic stripereader 601 include fraud prevention since a credit card can be scannedand fingerprints verified at the same time. Also, driver's licenses thatencode demographic data in the magnetic stripe can be read.

Sixth Embodiment

In a sixth preferred embodiment, illustrated in FIG. 7, the presentinvention further comprises at least one of a device-resident computerprocessor 701 and a device-resident non-volatile memory 203 for storingminutiae used for matching. Network access 219 to this memory 203 isprovided via the single tether 205 to the host computer 206 (see FIG.2). This memory 203, combined with the on-board computing power in thedevice, allows for a standalone matching system that can be updated overthe network via the tether.

Seventh Embodiment

In a seventh preferred embodiment, the device illustrated in FIG. 11 hasa magnesium extruded case 1101. Inside of this case, a prism bracket1103 holds the prism 1102 and light source 1104 so that the prism isflush with the surface of the case 1101 or extends slightly beyond thecase. The prism bracket is loaded with the light source 1104 by placingthe ears 1113 of the light source into the ear slots 1117 on the prismbracket 1103. After the light source has been placed, thin tabs (notshown) are screwed into tab holes 1118 so as to retain the light andpermit the light constrained but free movement for expansion andcontraction. After the tabs have been secured, the prism 1102 is slidinto the prism bracket so that the prism is flush with the tabs and theretaining sill on the prism bracket. The completed prism bracket is slidonto the extrusion so that the radiused edge 1116 of the bracket fitsinto the scallop 1124 of the case 1101. The prism bracket is swung upinto place using this hinge joint and the prism bracket is fastened tothe case 1101 using machine screws (not shown) in holes 1115,

The camera bracket 1122 has the lens threaded into the through hole1123. A board level camera is secured in place onto camera bracket 1122on the opposite side of the lens and the entire camera bracket assemblyis mounted onto the camera mount 1121 by screwing the camera bracket tothe camera mount through adjustment slot 1120. The inverter required forthe light pipe can be mounted on the front side of camera mount 1119 aswell. After the physical connection 1106 and the dichroic minor 1107have been mounted, the electrical connections within the system arecompleted and adjustments for the locations of the components are madeso as to ensure the camera is capturing the platen area properly.

In operation, light originating at the light source 1104 enters theprism 1102 and intersects the platen surface. Light, which totallyinternally reflects is directed toward the camera around the opticalaxis 1105. The light intersects a dichroic minor 1107 and reflectstoward the camera lens 1108. The light is then capture by the imagesensor 1109 on the camera unit 1110 and digitized as an image. Imagescaptured by the camera 1110 are transmitted over the connection 1106 toan attached computer.

Eighth Embodiment

In an eighth preferred embodiment, the device illustrated in FIG. 12 canhave the image sensor 1209 mounted at an angle that has been calculatedto minimize the depth of field requirement. FIG. 12 is otherwiseidentical to FIG. 11. As illustrated in FIG. 12, the body of the lens1108 remains parallel to the optical axis 1105 but, a board level camera1110 on which the image sensor 1209 is mounted is rotated by about threedegrees with respect to the perpendicular to the optical axis 1105. In adevice with a lateral magnification ratio of 1/13, a prism angle of 45degrees and a prism index of refraction of 1.52, the sensor 1209 shouldbe placed at an angle of 2.89 degrees.

Slip-Case

As illustrated in FIG. 9, a slip-case 900 is provided to cover theportable device when it is not in use. The slip-case 900 comprises apair of tabs 902 to lock into a corresponding pair of cutouts 901located in the handles 209 of the portable device. In a preferredembodiment, the slip-case is typically made of any suitably protectivehard polymer. Alternatively, the device is dip-coated with an elastomeror other protective polymer (not shown).

Case

As illustrated in FIG. 11, the device case can be manufactured as ametal or plastic extrusion. Alternatively, the case can be manufacturedusing a variety of processes including injection molding, Thixomolding,die casting, and investment casting. Thixomolding is a type of injectionmolding for metals. Previous approaches to manufacturing such a casehave not used extrusion since the resulting device size is too largeand/or the tolerances on the extrusion have not been good enough toyield a precision device. Typically, to be manufactured as an extrusion,the desired shape needs to be able to fit within a 12 inch diametercircle since material for creating the extrusion is delivered as at mosta 12 inch ingot (a cylinder of material). In this invention, since acompact size can be realized, an extrusion die can be made that createsthe main case of the device as a hollow tube within this 12 inchconstraint. This tube can then be machined and finished to final form.Since extrusions can be done with metals such as aluminum and magnesiumand various plastics, the use of an extrusion helps minimize the deviceweight due to material density and an ability to have thinner walls inthe case.

Method

FIGS. 8A-8D are a flow chart describing a preferred embodiment of themethod of the present invention. The host computer initializes 801 thecamera 204 before beginning any fingerprint capture session. This firststep establishes a camera connection during which the diagnostic data isread from the non-volatile data store 203. In addition, duringinitialization, deviant pixels are identified and aberration correction815 and perspective correction 815A have any required precomputed valuescreated. The system then waits for a request 802 that includes but isnot limited to: capture a fingerprint of a given type 803 et seq.,retrieve a captured image 830, retrieve diagnostic data 828-829, or endcapture session 831-833.

The capture fingerprint request 803 et seq. varies according to the typeof fingerprint being captured. Several capture types exist but all suchcaptures fit in either a roll or a slap format. In the following, acomposite image is defined as an image formed from one or more imageframes. Therefore, a composite image includes an output image createdfrom a sequence of scans from a linear sensor, a rolled print generatedfrom a series of images, or even a single slap image that was the resultof one processed image frame. These formats determine the processingsteps of the method to be performed. The different capture typestranslate to different camera configuration settings. Configurationsettings include area of platen to be captured, clock rate at whichpixels are captured, and camera exposure time.

When a capture fingerprint request is received 802, a new capturesession is initialized if an existing session does not already exist. Atthe beginning of each session, the camera 204 is configured forobtaining a reference offset image 803 and a reference offset image iscaptured 804. Next, the light source 201 is turned on under softwarecontrol 804. The system begins capturing frames from the camera 204 anda calibration process compares the current frame with the previousframe. The system calculates a metric that measures the change in theluminance of the light source 201 in the middle of each captured frame.When the change in luminance fall below a pre-set tolerance, i.e.,levels-off, and the rate of increase drops below a pre-set threshold,the light 201 is deemed to be “on”. Once the light 201 is turned on 805,luminance levels may be measured again and adjustments to the exposuresetting of the camera 204 are made until the luminance levels reach apre-set threshold level. Exposure adjustment is necessary as lightsource 201 brightness decreases over time. If increasing the exposurecannot compensate for the lack of luminance from the light source 201then the software reports that a maintenance check is required. If thesession initialized properly, the settings of the camera 204 forexposure, frame time and viewing window (based on the fingerprintcapture type) are set for a roll image 806 or a slap image 818. At thispoint, the system captures a blank frame from the camera 204 and keepsthis frame as a reference gain image for a roll image 807 or a slapimage 819. The process of capturing a roll or slap print now commences.

Roll Capture (FIG. 8B): Typical fingerprinting systems implement a footpedal 212, touch screen 211, mouse, a key on a keypad, or buttons 210 tobegin and end, i.e., “trigger”, the start and/or stop of a fingerprintcapture. Such switching mechanisms (mechanisms that signal a statechange) can be located on or in the device or external to the device ona computer, for instance. Embodiments of this invention may supportthese modes even using different modes to signal a beginning and an end.However, the preferred embodiments rely on an automatic or“self-generated” trigger that offers the end-users complete independencefrom physically manipulating other devices. This trigger, implemented inone of software, firmware, or hardware, eliminates the need for manuallysignaling the start and end of a roll. Triggers to start and/or stop afingerprint capture sequence are determined by statistics captured fromframe sequences obtained by the camera. These statistics measure framedirection and relative frame movement between successive frames obtainedby the camera to determine the type of fingerprint being captured. Oncethe type of fingerprint being captured is known and the device isinitialized for that type of fingerprint capture, the camera obtains asequence of frames and that sequence is analyzed for triggers. Theseautomatic triggers may be used in conjunction with the other existingswitching mechanisms described above.

In one embodiment, the process for initializing a roll occurs 808. Thesubject then positions the center of the finger on the platen so thatthe subject sees the fingerprint centered in the viewing window of theclient's user interface. The subject then rolls the finger in onedirection to the nail bed of the finger and then completely rolls thefinger in the opposite direction to the nail bed on the other side. Thefingerprint roll is complete and the fingerprint system returns to theclient software to let the subject know that the fingerprint roll iscomplete.

During this rolling process, the host computer 206 continuously capturesframes 809 from the camera 304. For each frame, the image ispreprocessed with offset and gain correction 810 before afingerprint-locating algorithm is applied. The fingerprint-locatingalgorithm analyzes the each frame for fingerprint data and if thesubject has placed a finger on the platen, then the system locates thefingerprint 811 in the frame and generates coordinates that describe abounding box around the fingerprint.

To compute the fingerprint location in each frame, two histograms aregenerated. The histograms are based upon image variance and arecalculated only on each row and column index that is evenly divisible bythe estimated fingerprint ridge width. The variance of the grayscalevalues of the pixels is calculated over an area roughly equal to thewidth of two ridges on the current frame for every pixel whose row andcolumn is evenly divisible by the estimated ridge width, and whose arearesides entirely within the current frame. If the variance of the pixelis greater than a pre-set threshold, then the associated positions ineach histogram are incremented. Once the two histograms have beengenerated, the first and last entries in each histogram that are above apre-set tolerance provide a rectangle encompassing the location of thefingerprint.

The automatic trigger process employs the current and previousfingerprint bounding box locations to determine finger travel distanceand direction between frames. The center column of each fingerprint iscalculated as the middle of the corresponding bounding box determined bystep 811. The centers of the current and previous locations are comparedto determine if the fingerprint is moving and if so, which direction thefinger is moving. If the Euclidian distance between the centers of thelocations is less than or equal to a predetermined number of pixels, thefingerprint is determined to be stopped. If the current frame center isgreater than a predetermined number of pixels right of the previousframe, the fingerprint is determined to be rolling right. If the currentframe center is greater than a predetermined number of pixels left ofthe previous frame, the fingerprint is determined to be rolling left.The predetermined number of pixels is typically at least about 10.

A half roll in one direction is started with a frame whose rolldirection is either left or right (the direction of the half roll). Thehalf roll is composed of a sequence of frames that have a direction ofeither stopped or direction of the half roll. The half roll is completedwhen the current frame's roll direction is opposite the direction of thehalf roll. If the half roll has a sufficient number of frames with aroll direction equal to the half roll direction, the full roll is begunand the capture sequence is started. Otherwise, the software returns towaiting for a half roll. The full roll is composed of a sequence offrames with roll directions opposite the direction of the half rolldirection, not including stopped. The full roll is completed when theroll direction of a frame is not equal to the direction of the full rollor a sufficient number of stationary frames have been captured. If asufficient number of frames are reached between the beginning of thefull roll and the end of the full roll, the software accepts the fullroll as complete. If the number of frames is insufficient, the systemcancels the full roll and returns to waiting for a half roll. If at anypoint during the rolls the finger is removed from the platen, thesoftware returns to waiting for a half roll.

When a roll starts or is cancelled, the composite image that representsthe fingerprint roll is initialized. As frames from the camera arecaptured, they are processed by applying offset and gain, fingerprintlocation, and trigger condition analysis. If the cancel condition isindicated then the current fingerprint roll is halted and the processreturns to the beginning of the fingerprint roll process 808. If,instead, the end roll condition is set then the composite image ispost-processed 814 814A 815 815A 815B. If there is no trigger conditionset then the current frame is merged into the composite image 813813.1-813.5 to create a composite roll image from a sequence of frames.The process of grabbing and processing frames continues in this manneruntil the roll end trigger occurs. The roll end trigger signals the endof the capture sequence.

For a roll, merging 813 into a composite image is done in five steps: 1)identifying where the current composite image and new fingerprint imageoverlap, 2) calculating the direction of the roll, 3) computing aninitial splice line by roughly aligning the fingerprint ridges betweenthe composite image and new fingerprint image, 4) use a quality metricto refine the splice line from the top to the bottom of the image, 5)combine the new image frame into the merged composite image usingmorphing along the splice line.

Overlap area 813.1: The overlap area between the merged composite imageand the new fingerprint image has been described above.

Roll direction 813.2: The direction of the roll can be determined bycomputing which side the new fingerprint image is located. For example,if the new fingerprint image is located on the left side of thecomposite image then the roll direction is to the left.

Initial splice line 813.3: Create an initial splice line based on theendpoints where the new fingerprint image and the composite fingerprintimage intersect in the overlap area then compute the slope of this newsplice line segment. Compute the center of this new splice line segment.Determine which two fingerprint ridges, near the center of the new imageand the merged composite image, have the best alignment. A metric thatcan be used is the local gray scale average along the splice line. Thiscenter location of the splice line is updated so to this identified bestmatch point so that splice line refinement can occur at this reliableanchor point. Copy this new splice line and its center location and callit the composite splice line.

Splice line refinement 813.4: Starting from the center of the new spliceline segment iterate up the splice line segment a pre-determined numberof pixels at a time. The starting position and the ending position onthe splice line identify an area of the splice line that is beingrefined. In the top row of this refinement region, iterate from athreshold number of pixels left of the splice point to a thresholdnumber of pixels to the right of the splice point. Form a splice linesegment candidate from the iterated pixel to the starting position onthe splice line. Compute a common similarity metric between allcandidate splice line segments so formed. One similarity metric computesthe pixel intensity average of the two areas and compares the averages.The result of the comparison is a score that represents how close thesetwo areas match, which represents how well the ridges line up. Once allthe comparisons are done, the best refined pixel location becomes thenew point on the composite splice line for this particular row. Thisprocess iterates to the top of the overlap area and from the center ofthe splice line segment to the bottom of the overlap area. The result isa final composite splice line based on the initial splice line.

Morphing (merging) into composite image 813.5: The existing compositeimage and new fingerprint image form a new composite image. The initialsplice line and composite splice line control what region of thecomposite image gets blended with the new fingerprint image region.Iterate from the bottom of the overlap region to the top of the overlapregion along both splice lines simultaneously. For each row consider thepixels on that row between the splice lines and a pre-determinedthreshold number of pixels outside of the splice lines. Iterate acrossthis interval of pixels on the row and assign the value of the compositeimage at that location as a value weighted by distance between the twosplice lines. Thus, data from the merged composite image is morphed intothe data from the new fingerprint image.

The merging method to create the rolled fingerprint from a sequence offrames typically comprises the following steps. The first frame iscopied to the composite image. The bounding boxes, located as describedabove for locate print 811, for the current and previous fingerprintlocations are intersected to form an overlap area. The left of theoverlap area is equal to the maximum of the left coordinates of the twobounding boxes. The top of the overlap area is equal to the maximum ofthe top coordinates of the two bounding boxes. The right of the overlaparea is equal to the minimum of the right coordinates of the twobounding boxes. The bottom of the overlap area is equal to the minimumof the bottom coordinates of the two bounding boxes. The center columnsof the overlap area in the current frame and composite image areexamined to find where fingerprint ridges intersect the columns. Theseintersections are compared between the new frame and the composite imageand they are used to perform a dynamic stretch on the current frame. Ifthe current frame and composite image are too dissimilar then themerging is aborted and the subject is warned that the finger is movingtoo drastically. In this case, a new roll capture is automaticallystarted. The current frame is stretched so that the ridges roughlyintersect with the existing ridges from the composite image and thecurrent image is morphed with the composite image to produce a newcomposite image. The final composite image becomes the fingerprint rollimage.

Typically, adjacent opposed dark edges of two sequential bounding boxesare compared by taking a histogram to analyze each dark edge and amatching algorithm is used to match ridges to obtain an image such thatthe fingerprint ridges are continuous.

Image processing techniques remove image defects introduced by thecamera and the lens. Six main processing steps occur: deviant pixelcorrection, offset and gain correction, high pass filtering, aberrationcorrection, perspective correction, and noise filtering.

Offset and gain correction 810: Applying offset and gain is a pixel bypixel operation. For each pixel, a lowest and highest acceptable valueare defined by the offset and gain images respectively. Offset and gaincorrections stretch these lowest and highest values to the maximum rangeallowed in a pixel. Therefore, a mapping between the observed interval(from the offset and gain pixel) is made to the maximum range output.Using an observed pixel in an image as an input to this mapping yieldsan output pixel that has been offset and gain corrected. Input pixelsbelow the lowest limit or above the highest limit are saturated at therespective values on output.

Deviant pixel correction 814: Most cameras contain imperfections on thesensory chip. These imperfections manifest themselves as intensityvalues that differ greatly from the normal expected intensity values.These deviant pixels are beyond a threshold away from the averageintensity value of a neighborhood of pixels.

Deviant pixel correction involves two steps. The first step involvesinitializing the deviant pixel subsystem. The second step is thecorrection of individual frames.

Initialization, which occurs in the session initialization 801, requiresan offset image acquired in camera initialization. The grayscale valueof each pixel in the image below the second row and above the second tolast row is compared to an average grayscale value of the two pixelsabove and two pixels below the current pixel. If the grayscale value ofthe current pixel significantly differs from the average of the otherfour pixels, the current pixel's location is added to a cached list foruse in the second step.

Deviant pixel correction of the individual frames is relatively simple.For each pixel location cached in the first step, the system averagesthe grayscale values of the two pixels above and two pixels below andreplaces the grayscale value of the current pixel with that average.

High pass filtering 814A: Edge details of the fingerprint ridges may beenhanced at the cost of increasing noise in the image frame. An approachto enhancing the edges is applying a high pass filter as is commonlyknown in the image processing field. Such filters convolve a high passfilter with the image. Typical high pass filters used may have a kernelsize of 3, 5, or 7. The strength of the high-pass filter being used isdriven by the application requirements.

Aberration correction 815: The camera lens also introduces image defectssuch as pincushion or barrel distortions. Parameters for correction ofthese defects are identified during calibration at the time of devicemanufacture and these parameters are stored in the non-volatile memory203. In camera initialization, these parameters are read and the defectmodel is initialized.

The output coefficients and the defect model are used to correct acaptured image using an algorithm such as the Ojanen algorithm.Correction amounts to local averaging in neighborhoods defined by thedefect model and the output coefficients. Alternatively, aninterpolation method such as bi-linear interpolation or nearest neighborcan be used to create an output pixel for each neighborhood.

Perspective correction 815A: Perspective correction of the image mayalso be performed. In using an algorithm such as the Ojanen algorithm, amathematical description of the perspective model identified by thealgorithm can be used in conjunction with an interpolation algorithmsuch as bi-linear interpolation to generate the final correctedcomposite image. Correction in the image occurs after the final roll orslap has been captured. Alternatively, if the device is made in aprecision fashion, the perspective correction can be geometricallymodeled as a three-dimensional relationship between planes. Oncemeasured, the mathematical description of these planes can be used inconjunction with bi-linear interpolation to create a perspectivecorrected composite image. In both cases described here, theinitialization step 801 preferentially precomputes the perspectivecorrection parameters for each pixel so that in full operation extratime would not have to be spent on calculating needed weightsrepeatedly.

Noise filtering 815B: The noise filter algorithm convolves the imagewith an averaging convolution kernel. In the convolution operation, thevariance within the neighborhood is used in conjunction with a fixedthreshold. If the variance exceeds the threshold then the original pixelis left unchanged otherwise, the pixel is assigned the value of theconvolution. The convolution kernel size is established experimentallyaccording to an application's requirements.

When post-processing is completed the diagnostic values are updated 816.When all the statistical data has been written back into thenon-volatile memory 203, control returns to the host program. The hostprogram then saves the final image and returns to and signals completion817. At this point, the host software requests the image just captured,requests another fingerprint acquisition, reads non-volatile memory, orcloses its session.

In alternative embodiments of capturing the rolls, the image processingsteps may have their order changed and merging of the composite rolledimage may occur at any stage of processing the frame.

Slap Capture (FIG. 8C): When acquiring a slap print, similar processingsteps to that of the roll capture are performed. From the subject'sperspective the subject places the target fingers or thumb on theplaten. Frames from the camera are continuously captured and processeduntil the trigger condition indicates a good slap capture. For eachframe captured, offset and gain correction are applied and the frame isanalyzed for the presence of a trigger condition. This analysis involvescalculating the variance of sub windows within the full frame. Each subwindow is square with the length of the sides roughly equal to the widthof two fingerprint ridges. Sub windows are centered on every pixel whoserow and column index is evenly divisible by the ridge width and whosesub window area resides entirely within the current frame. If a pixel'svariance is greater than a certain threshold then a count is incrementedand the same operation is performed on the previous image in the samelocation. If the pixel's variance in the previous image is also greaterthan the threshold, a second count is also incremented. The ratio of thenumber of pixels that are above the variance threshold in both images tothe number of pixels that are above the variance threshold in only thecurrent image is used to determine how similar the two images are. Ifthe images are similar enough for a small sequence of a few frames, thecurrent frame has the capture condition set and the best frame (frames)is (are) saved.

If a capture image trigger condition did not occur then the process ofcapturing and processing frames continues 821 to 824. The final capturedframe is post processed 825 825A 826 826A 826B as described above forroll capture 814 814A 815 815A 815B. When the post processing iscomplete the software updates diagnostic data in non-volatile memory827, saves the image and indicates the capture is complete 828 to thehost program and processes a new request 802.

In alternative embodiments of capturing the slaps, the image processingsteps may have their order changed and merging of the composite rolledimage may occur at any stage of processing the frame.

The present invention applies to large formatfingerprint/handprint/footprint scanning as well as pet imagingapplications. Other extensions of this technology to other applicationsare also possible. The descriptions herein are not meant to be limitingto the applications described herein but were intended to beillustrative of the application of this invention. For instance, thesame invention can be applied in newborn applications in which thefootprints of newborns are digitally captured, in applicant processingapplications for capture and submission of fingerprints for criminalbackground purposes, for arrestee or detainee applications, forverification of the person's identity when the collected prints arematched against a database of prints, for access control applications,and for applications to sampling pet paws or pet noses to maintainidentities of animals with certified pedigrees. Many times demographicdata must be collected in conjunction with the fingerprints so that thefingerprints can be associated with a name, address, identificationnumber, etc. Such demographic data allows expedited matching in manydatabase systems.

1. A system configured to generate a composite image of a friction ridgesignature of a subject, the system comprising: a platen comprising afirst light-reflecting surface; a light source configured to emit lightrays to illuminate a subject placed in contact with the firstlight-reflecting surface of the platen; a camera configured to captureimages of the subject in contact with the first light-reflecting surfaceof the platen, through the platen, in an ongoing manner; and a processorconfigured to receive images of the subject captured by the camera, andto automatically determine whether a sequence of the received imagesincludes a successful capture sequence, wherein a determination that thesequence of received images includes a successful capture sequence ismade responsive to identification by the processor of a trigger in thesequence of received images indicating the beginning of a capturesequence and identification by the processor of a trigger in thesequence of received images indicating the completion of the capturesequence.
 2. The system of claim 1, wherein processor is configured todetermine a type of capture sequence, and to determine whether thesequence of received images includes a successful capture sequence ofthe determined type.
 3. The system of claim 2, wherein the type ofcapture sequence comprises a roll capture or a slap capture.
 4. Thesystem of claim 1, wherein the processor is configured to generate,responsive to determination that the sequence of received imagesincludes a successful capture sequence, a composite image, wherein thecomposite image is formed from the images included in the successfulcapture sequence.
 5. The system of claim 4, wherein the processor isconfigured to generate the composite image by merging the imagesincluded in the successful capture sequence.
 6. The system of claim 1,wherein the processor is configured to identify the trigger indicatingthe beginning of the capture sequence by comparing successive images inthe sequence of received images.
 7. The system of claim 1, wherein theprocessor is configured to identify the trigger indicating thecompletion of the capture sequence by comparing images in the sequenceof received images that were captured subsequent to the triggerindicating the beginning of the capture sequence.
 8. A method ofgenerating a composite image of a friction ridge signature of a subject,the method comprising: emitting light rays toward a platen comprising afirst light-reflecting surface to illuminate a subject placed in contactwith the first light-reflecting surface of the platen; capturing imagesof the subject in contact with the first light-reflecting surface of theplaten, through the platen, in an ongoing manner; and executing one ormore modules on one or more processors to automatically determinewhether a sequence of the received images includes a successful capturesequence, wherein a determination that the sequence of received imagesincludes a successful capture sequence is made responsive toidentification by the processor of a trigger in the sequence of receivedimages indicating the beginning of a capture sequence and identificationby the processor of a trigger in the sequence of received imagesindicating the completion of the capture sequence.
 9. The method ofclaim 8, wherein determining whether a sequence of the received imagesincludes a successful capture sequence comprises: determining a type ofcapture sequence, and determining whether the sequence of receivedimages includes a successful capture sequence of the determined type.10. The method of claim 9, wherein the type of capture sequencecomprises a roll capture or a slap capture.
 11. The method of claim 8,further comprising generating, responsive to determination that thesequence of received images includes a successful capture sequence, acomposite image, wherein the composite image is formed from the imagesincluded in the successful capture sequence.
 12. The method of claim 11,wherein the composite image is generated by merging the images includedin the successful capture sequence.
 13. The method of claim 8, whereinidentification of the trigger indicating the beginning of the capturesequence is performed by comparing successive images in the sequence ofreceived images.
 14. The method of claim 8, wherein identification ofthe trigger indicating the completion of the capture sequence isperformed by comparing images in the sequence of received images thatwere captured subsequent to the trigger indicating the beginning of thecapture sequence.